Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection

Cancers can have highly heterogeneous uptake patterns best visualised in positron emission tomography. These are essential to detect, diagnose, stage and predict the evolution of cancer. Due this heterogeneity, a general-purpose cancer detection model be built using unsupervised learning anomaly models; these models learn healthy representation tissue detect by predicting deviations from appearances. This task alone requires capable accurately long-range interactions between organs, imaging patterns, other abstract features with high levels expressivity. Such characteristics suitably satisfied transformers, been shown generate state-of-the-art results training on data. work expands upon such approaches introducing multi-modal conditioning transformer via cross-attention, i.e. supplying anatomical reference information paired CT images aid PET task. Using 83 whole-body PET/CT samples containing various types, we show that our method is robust achieving accurate localisation even cases where data unavailable. Furthermore, proposed uncertainty, conjunction kernel density estimation approach, provide statistically alternative residual-based maps. Overall, superior performance demonstrated against leading alternatives, drawing attention potential approaches.